1. Field of the Invention
This invention relates to the field of mass storage systems. In particular, the invention relates to the management of storage transactions in, and the configuration of, intelligent storage area networks for the purposes of allocating and changing the allocation of storage resources.
2. Description of the Related Art
The storage of large amounts of data in so-called mass storage systems is becoming a common practice. Mass storage systems typically include storage devices coupled to file servers on data networks. Users in the network communicate with the file servers for access to the data. The file servers are typically connected to specific storage devices via data channels. The data channels are usually implemented with point-to-point communication protocols designed for managing storage transactions.
As the amount of storage increases, and the number of file servers in communication networks grows, the concept of a storage area network (SAN) has arisen. Storage area networks connect a number of mass storage systems in a communication network which is optimized for storage transactions. For example, Fibre Channel arbitrated loop (FC-AL) networks are being implemented as SANs. The SANs support many point-to-point communication sessions between users of the storage systems and the specific storage systems on the SAN.
File servers and other users of the storage systems are configured to communicate with specific storage media. As the storage systems expand or media is replaced in the system, re-configuration is required at the file servers and other users. Also, if a need arises to move a data set from one device to another, in a so-called data migration operation, it is often necessary to block access to the data set during the migration process. After migration is complete, re-configuration at the user system must be executed to make the data available from the new device. The blocking of access to the data set during the transfer is a particularly costly action for large data sets having many users. Both the amount of time required to move a copy of the data to the new device, and the number of people inconvenienced can be very large.
Overall, as the complexity and size of storage systems and networks increase, the problems of managing configuration of the users of the data and of the storage systems themselves multiply. Accordingly, there is a need for systems that simplify management of storage systems, and in particular the migration of data from one device to another, while taking advantage of the flexibility and power of the SAN architecture.
The present invention provides a method and apparatus for use in a storage network that facilitates the transfer of a data set from a first storage device to a second storage device, without blocking access to the data set during the transfer. In one embodiment, an intermediate device for the storage network is provided. The intermediate device comprises a plurality of communication interfaces, including at least a first interface adapted for communication with the first storage device which stores the data set, a second interface adapted for communication with a second storage device, and a third interface adapted for communication with a client processor. Data transfer resources are coupled to the plurality interfaces, and are used for transferring data access requests identifying the data set from the third interface to at least one of the first and second interfaces. A logic engine within the intermediate device is coupled to the first and second interfaces, and is responsive to a control signal to manage the transfer of the data set through the first interface from the first storage device, and through the second interface to the second storage device without blocking the transfer data access requests from the client processor.
According to various embodiments of the invention, the logic engine comprises data structures that store information, such as status information, information identifying the data set, and other data concerning the transfer. In one embodiment, the intermediate device stores a parameter indicating an extent of the data set which is already copied to the second storage device.
According to other aspects of the invention, the data transfer resources include logic operable during the transfer of the data set which direct data access requests to the first and second storage devices in response to a type of data access request, and a status of the transfer. In one embodiment, when the data access request comprises a request to write data in the data set, the data transfer resources direct the data access request to both the first and second storage devices if the request identifies data already copied to the second device. In another embodiment, when the data access request comprises a request to read data in the data set, the data transfer resources include logic to direct the data access request to one or both of the first and second storage devices.
In one embodiment of the invention, logic is included to assign a priority to the transfer in relation to data access requests from the client processor, and to manage the transfer in response to the priority. For example, a high priority is set to the data access requests relative to the transfer. In this manner, the client processor does not experience significant delays obtaining fulfillment of its data access requests during the transfer.
According to other embodiments of the invention, the data transfer resources comprise a plurality of driver modules, and configurable logic linking driver modules into data paths, so that data paths include respective sets of driver modules. The plurality of driver modules includes one or more hardware driver modules to managers of communication interfaces, and one or more internal driver modules to perform data path tasks independently of the plurality of communication interfaces.
According to another embodiment of the invention, the data transfer resources comprise redundant data paths supporting reliability and substantially continuous operation.
According to another embodiment of the invention, a user interface is provided on the intermediate device, providing input by which control signals are supplied to control the hot copy process.
The present invention also provides a method for copying a data set in use by a client, and stored in the first device, and supplying the copy to the second device. The method includes placing an intermediate device in a path between the client and first and second storage devices. The method also includes a step of transferring copies of blocks of data in the data set from the first device to the second device through the intermediate device, until a copy of the data set is stored in the second device. During the transferring of the copies of blocks, the method provides for transferring data access requests having one of a plurality of types, from the client through the intermediate device to at least one of the first and second devices in response to progress of the transfer of the copies of blocks of data in the data set, and in response to the type of data access request.
According to one embodiment of the invention, the step of transferring copies of blocks includes:
(i) setting a parameter indicating a size and location of the data set;
(ii) generating a request to copy a block from the data set to a buffer in the intermediate device;
(iii) generating a request to transfer the block from the buffer to the second device;
(iv) setting a parameter indicating blocks from the data set stored in the second device; and
(v) repeating the steps (ii) through (iv), until a copy of the data set is stored in the second device.
In one embodiment of the invention, the method includes fulfilling the data access requests through the intermediate device.
In another embodiment of the method, the step of transferring copies of blocks comprises a background process executed without blocking data access requests from the client.
According to yet another embodiment, the present invention provides a storage server having sophisticated data processing resources for the purposes of controlling the routing of data storage transactions, and the migration of data sets among a pool of storage devices. The data processing resources comprise a plurality of driver modules and configurable logic linking driver modules into data paths. Each configured data path acts as a virtual circuit that includes a set of driver modules selected from the plurality of driver modules. A data storage transaction which is received at a communication interface is mapped to one of the configured data paths according to the initiating host and according to the logical address of the storage extent subject of the transaction. Upon completion of a hot copy process, the configuration of the data path is changed to direct the session including the transaction to the new location of the data. During the transferring, the hot copy process manages routing of the reads and writes in the data path according to the status of the hot copy process.
According to another aspect of the invention, the plurality of driver modules includes a protocol server for a protocol supported on a communication interface in the plurality of communication interfaces. The protocol server recognizes target identifiers, or logical addresses, which identify particular storage extents according to the protocol on the interface. Transactions addressed to the particular storage extent are mapped to a particular configured data path in the server.
The data paths configured in this manner act as virtual storage devices. Users of the data communicate with a communication interface on the storage server according to a protocol for a particular storage device. Inside the server, the transactions according to that protocol are mapped to a virtual storage device implemented by sets of drivers. Setting up and changing the storage tasks performed in a particular data path, and setting up and changing the mapping of a storage extent from one data path to another are accomplished by configuring the sets of driver modules within the storage server.
The present invention provides an intelligent storage routing device which manages logical and physical access to a pool of shared storage devices, and transfer of data sets among the storage devices without blocking access to the data sets. The device is logically closer to the client server than to the storage devices, and responds to client specific requests for storage transactions which require accessing and sharing a storage area network coupled to the device. The device manages the use of a cache memory to enhance performance. Very large data sets, on the order of many terabytes in size or more, can be transferred from old to new storage devices, or vice versa, as the storage resources in a network change, without blocking access to the data sets.
Other aspects and advantages of the present invention can be seen upon review of the figures, the detailed description and the claims which follow.